This open access book provides a comprehensive overview of the state of the art in research and applications of Foundation Models and is intended for readers familiar with basic Natural Language Processing (NLP) concepts. Over the recent years, a revolutionary new paradigm has been developed for training models for NLP. These models are first pre-trained on large collections of text documents to acquire general syntactic knowledge and semantic information. Then, they are fine-tuned for specific tasks, which they can often solve with superhuman accuracy. When the models are large enough, they can be instructed by prompts to solve new tasks without any fine-tuning. Moreover, they can be applied to a wide range of different media and problem domains, ranging from image and video processing to robot control learning. Because they provide a blueprint for solving many tasks in artificial intelligence, they have been called Foundation Models. After a brief introduction to basic NLP models the main pre-trained language models BERT, GPT and sequence-to-sequence transformer are described, as well as the concepts of self-attention and context-sensitive embedding. Then, different approaches to improving these models are discussed, such as expanding the pre-training criteria, increasing the length of input texts, or including extra knowledge. An overview of the best-performing models for about twenty application areas is then presented, e.g., question answering, translation, story generation, dialog systems, generating images from text, etc. For each application area, the strengths and weaknesses of current models are discussed, and an outlook on further developments is given. In addition, links are provided to freely available program code. A concluding chapter summarizes the economic opportunities, mitigation of risks, and potential developments of AI.

BANGKOK/LONDON – A week after the Myanmar military seized power, a Twitter account that had lain dormant for nearly a decade flickered back into life. The Twitter user mocked anti-coup protesters, hundreds of whom have been killed in a crackdown by security forces since the Feb. 1 coup. After a police truck fired high-pressure water cannons on demonstrators in the capital city of Naypyidaw on Feb. 8, he made a trolling reference to the nation's traditional April new year celebration: "Water festival come earlier for them lol." A few weeks later, the user wrote "#fuckthereds," making a dismissive reference to the political party of Aung San Suu Kyi, the Nobel Prize-winning civilian leader who had been overthrown and arrested in the coup. A review of an archived version of the account, which has since been shut down, revealed the username was a pseudonym belonging to Ivan Htet, the 33-year-old son of a leading figure in the coup: the chief of the air force, Maung Maung Kyaw. But Ivan Htet hasn't just been an enthusiastic supporter on social media of the Tatmadaw, the name for the Myanmar military, which has dominated political life since independence in 1948 for Myanmar, then called Burma. He is also trying to cash in, helping equip the military, along with his wife Lin Lett Thiri, who co-founded a private firm to supply Myanmar's armed forces, Reuters has found.

Existing question answering datasets focus on dealing with homogeneous information, based either only on text or KB/Table information alone. However, as human knowledge is distributed over heterogeneous forms, using homogeneous information might lead to severe coverage problems. To fill in the gap, we present \dataset, a new large-scale question-answering dataset that requires reasoning on heterogeneous information. Each question is aligned with a structured Wikipedia table and multiple free-form corpora linked with the entities in the table. The questions are designed to aggregate both tabular information and text information, i.e. lack of either form would render the question unanswerable. We test with three different models: 1) table-only model. 2) text-only model. 3) a hybrid model \model which combines both table and textual information to build a reasoning path towards the answer. The experimental results show that the first two baselines obtain compromised scores below 20\%, while \model significantly boosts EM score to over 50\%, which proves the necessity to aggregate both structure and unstructured information in \dataset. However, \model's score is still far behind human performance, hence we believe \dataset to an ideal and challenging benchmark to study question answering under heterogeneous information. The dataset and code are available at \url{https://github.com/wenhuchen/HybridQA}.